Security is a concern in a variety of transactions involving private information. As an example, iris recognition is a well-accepted and accurate means of biometric identification used in government and commercial systems around the world that enables secure transactions and an added layer of security beyond keys and/or passwords. Due to the increased security provided by iris recognition systems, an increase in use of such systems has occurred around the world.
Traditional cameras used in biometric identification are generally expensive. Video cameras that use complementary metal-oxide-semiconductor (CMOS) or semiconductor charge-coupled device (CCD) image sensors typically use electronic shutters to determine the time period over which the sensor measures light. A trigger signal opens the shutter for a predetermined time during which each pixel within the sensor array collects (integrates) incoming light. At the end of the exposure, the signal collected (integrated) in each pixel during the exposure remains fixed and is then systematically read out, converted to a digital signal, and processed to become an image. Pixels are then cleared and readied for the next exposure to light.
There are different types of electronic shutters used in the industry. FIG. 1 is a diagram of a traditional global shutter camera schedule of events for signal collection, including the timeline for exposure of each row of an image sensor in the global shutter camera. Global shutters simultaneously expose an entire array of pixels, e.g., N rows by M columns, during signal collection. During a single exposure event (during texp), light is simultaneously collected in each pixel. When the global shutter closes, the light signal within each pixel represents the image during the period of the single exposure. All pixels integrate signal over exactly the same period of time. Global shutter cameras avoid flash timing issues incurred when using rolling shutter cameras. However, global shutter cameras are expensive options for biometric analysis, thereby increasing the overall costs associated with biometric analysis systems.
Rolling shutter cameras save cost and size in their sensor design. FIG. 2 is a diagram of a traditional rolling shutter schedule of events for signal collection, including the rolling shutter timeline. Rolling shutters expose an array of pixels differently from global shutters. A rolling shutter system exposes a first row of pixels for an exposure time (texp) and then commences to read-out the exposed row of pixels for digitization. The read-out process occupies a unique onboard resource for a period referred to as a read-out time during which no other row can be read-out. To minimize the duration of the total exposure including the read-out process, the rolling shutter exposes the second row of pixels during a time that is equal to but delayed from the first row by a read-out time. The second row is thereby exposed to light and ready to be read-out at the moment that the read-out process for the first row is complete. The third row is exposed to light for a time interval equal in length to that of the first two rows but delayed relative to the second row by a read-out time allowing for the required time to read-out the second row. The process “rolls” down the pixel array reading row-by-row in sequence taking a total time equal to the exposure time for a single row plus the read-out time interval, times the number of rows. The time interval during which a row is exposed and therefore the events captured by that row are different for each row for a rolling shutter sensor. This is a key difference from a global shutter sensor, especially when using a short flash.
As shown in FIGS. 1 and 2, the light collection time period for each row of a sensor with a global shutter is simultaneous while the time periods of light collection for each row of a sensor equipped with a rolling shutter are not simultaneous. Rather, light collection time periods for each row of a rolling shutter are offset from one another with a delay between rows equal to the row read-out time. The different exposure techniques result in image shearing. For example, FIG. 3 shows an image in which a moving fan blade was captured by an image sensor with a global shutter with no or little distortion as compared to the same moving fan captured by an image sensor with a rolling shutter shown in FIG. 4. Image shearing is an inevitable consequence of the row-by-row time delays built into a rolling shutter sensor in which each row “sees” the scene over a slightly different and offset time interval.
The row-by-row delay in exposure of a rolling shutter also has an effect on coordinating an exposure with flash illumination. As discussed herein, a flash refers to a short, intense period of illumination of a subject during which the light applied by the flash dominates other sources of light on the scene (e.g., the area surrounding the subject). FIG. 5 shows the exposure of each row during texp with the shaded region indicating the duration of time of the flash illumination occurring simultaneous during texp. In a global shutter, the exposure of all the pixels in a sensor can be coordinated with the application of the flash illumination to the scene. For example, if the period of the flash pulse is 1 ms, the global shutter can open simultaneously with the start of the pulse and close simultaneously with the end of the pulse 1 ms later. The flash illuminates the pixels during and only during their global exposure. Light forming the image is, by assumption, dominated by the light applied to the scene by the flash. If, for example, sunlight is present in the scene, the irradiance on the object from the flash is significantly brighter than that of the sunlight during the flash pulse when pixels are exposed by the global shutter.
Coordinating a flash pulse with a rolling shutter exposure is more complicated than with a global shutter. FIG. 6 shows the rolling shutter exposure during texp for each row extending diagonally across the diagram, and the time period for flash illumination illustrated as the vertical shaded region tp. After a flash delay, the flash pulse occurs between the start pulse and end pulse points of the diagram. Because the exposure period for each row is delayed from the previous row by a short read-out time interval, illumination of a full frame requires that a flash pulse provide illumination during a period when all rows are integrating light. Failure to meet this condition creates a situation in which some rows of pixels integrate light from the flash pulse while some do not, and perhaps some rows integrate light from only a portion of the flash pulse. In this case, the image is unevenly illuminated. As shown in FIG. 6, some rows are finished integrating before the flash starts and other rows do not start integrating until after the flash ends. In addition, other rows integrate a partial flash and some integrate the full flash. Thus, a subset of lines on the rolling shutter sensor receive adequate illumination, but outside of this set of lines, the other parts of the sensor remain largely dark.
One example of a short flash pulse can be considered with respect to FIG. 6, which, across the top horizontal line, four horizontal dashed lines, and bottom lines, respectively shows row numbers 0, 200, 450, 600, 850, and 1000. The flash pulse can start as row 200 of 1000 rows finishes integrating the signal and begins to read-out, indicated by the top dashed line. The flash pulse can end as row 850 of 1000 begins integrating the signal, indicated by the bottom dashed line. FIG. 6 shows that rows 450 through 599 receive the full illumination of the flash pulse, as bracketed by the middle two dashed lines. However, rows 200 to 449 and rows 600 to 849 only receive a portion of the flash illumination while rows outside of these ranges, e.g., rows 1 to 199 and 850 to 1000, receive no flash illumination. Assuming insignificant ambient light, the resulting image would show an illumination stripe surrounded by dim regions. The transition from bright to dim at the top and bottom of the stripe is due to rows that receive flash illumination over a fraction of the total pulse time. FIG. 7 shown a portion of an image acquired using a rolling shutter camera with a delayed flash pulse in which the recorded irradiance is plotted to show dark regions before and after the flash, ramp-up and ramp-down regions of partial illumination, and a plateau region of complete flash illumination. The plateau region of FIG. 7 is not flat because the flash itself was not uniform over the field-of-view. As another example, an image captured using a rolling shutter camera would include a horizontal stripe with a vertical height proportional to the duration of the flash illumination. In cases with bright ambient illumination, the un-flashed portion of the image would appear, but might be significantly dimmer if the flash illumination is brighter than the ambient illumination.
When an image of a particular object is desired with a rolling shutter camera, a trigger signal can be initiated by the sensor controller to fire the flash at a preset time relative to the start of image writing. For example, the flash can fire when the first line is written and can remain on for 50 of 1000 lines. The resultant image would be flash illuminated for the top 5% of the image and would be dark elsewhere. The same flash can be delayed until the 500th line of 1000 lines, resulting in an image with a stripe of illuminated content approximately halfway down the frame. With such an arrangement, the photographer would need to align the subject within the camera field-of-view such that the stripe of illumination detected by the sensor corresponds to the position of the desired object.
Traditionally, one solution to the problem of flash illuminating an image using a rolling shutter has been to use an extended period of illumination, e.g., a flash pulse that is started simultaneously with the beginning of the exposure of the first row of pixels and is not finished until the last row of pixels has been exposed. The extended period of illumination is needed to expose the entire image since image lines are written sequentially rather than all at once (as is the case with a camera including a more expensive and physically larger global shutter sensor). This technique necessitates a longer flash pulse compared to the global shutter case, and would show up in FIG. 6 as a shaded region covering all of the rows with a duration equal to the frame time. This technique would also illuminate the full frame shown in FIG. 7. Additional requirements on the flash in terms of power output, heating and reliability are needed based on the longer pulse for this technique. A longer pulse might also challenge requirements for eye-safety. For these reasons, full frame pulses with rolling shutters are considered impractical.
Thus, a need exists for improved biometric analysis systems including a rolling shutter that are capable of illuminating and capturing the desired area of an object for identification without an extended flash. These and other needs are addressed by the systems and methods of biometric analysis of the present disclosure.